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Abstract 

In this paper we investigate the value of collaboration scripts for promoting 
metacognitive knowledge in a project-based e-learning context. In an empiri- 
cal study, 82 students worked individually and in groups on a project using 
the e-learning environment MyProject, in which the life cycle of a project is in- 
herent. Students followed a particular collaboration script that combines indi- 
vidual and collaborative activities, aiming to promote individual and socially 
shared reflective thinking during the planning and evaluation phases of the 
project. We analysed group discussions and evaluation questionnaires, and 
the results provide evidence about the importance of the design variables con- 
sidered in the collaboration script for cultivating metacognitive knowledge, 
such as project phase, roles undertaken by students, degree and type of inter- 
action, type of activities and products, and activity sequencing. (Keywords: 
Project-based learning, e-learning, collaboration, peer interaction, metacog- 
nitive knowledge, asynchronous discussions) 


P roject-based learning (PBL) is rooted in the idea that a problem or 
question drives learning activities toward the construction of a con- 
crete artifact in an authentic context. In this process, students pursue 
solutions to open-ended problems by formulating questions for investiga- 
tion; designing plans or proposals; collecting, analyzing, and integrating 
information; constructing explanations and models; and creating artifacts 
or products of their understanding (Blumenfeld, Soloway, Marx, Krajcik, 
Guzdial, & Palincsar, 1991). It is also essential that learners have the oppor- 
tunity to control the learning process; make decisions regarding the pacing, 
sequencing, and content of learning; and evaluate the outcome of their ef- 
forts and their own learning strategies. Such a learning context involves both 
vertical learning (referring to accumulation of the subject matter knowl- 
edge) and horizontal learning (referring to generic skills, such as project 
management and collaboration) (Helle, Tynjala, & Olkinuora, 2006). 
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The inherent openness of project-based environments seems to introduce 
more cognitively complex tasks in the learning process, making experience, 
self-direction, and high-level metacognitive control necessary for learners to 
effectively undertake PBL (Barron, Schwartz, Vye, Moore, Petrosino, Zech, 

& Bransford, 1998; Jonassen, 1999; Land & Zembal-Saul, 2003). Although 
the strength of PBL seems to lie in giving students the opportunity and mo- 
tive to work in a personally meaningful way toward a “solution,” several re- 
searchers observe students’ deficiencies in performing the various activities 
in PBL, referring to self-directed learning skills and metacognitive knowl- 
edge (Bereiter & Scardamalia, 1993; Hannafin & Land, 1997; McLoughlin & 
Hollingworth, 2001; Schank & Cleave, 1995; Thomas, 2000). For example, 
novice learners often exhibit an absence of prior knowledge, whereas in- 
experienced and young problem solvers lack essential metacognitive skills, 
such as planning and self-monitoring, as well as a repertoire of learning 
strategies necessary for engaging in independent learning. Thus, strengthen- 
ing students’ metacognitive and reflective skills is essential in such innova- 
tive learning environments to assist them in adopting the strategies and 
reflective processes that will enable them to define, plan, and self-monitor 
their thinking and learning. To this end, computer-based tools have been 
used to help learners organize and reflect on their work and to support them 
in sharing and making their thinking more explicit and “visible” (Azevedo 
& Hadwin, 2005; Land & Greene, 2000; Linn, 2000; Quintana, Zhang, & 
Krajcik, 2005; White, Shimoda, & Frederiksen, 2000). 

Especially in the area of PBL, a number of contemporary Web-based 
learning environments have been developed recently that support learners in 
dealing with the open issues of a project at the individual or group level. KBS 
Hyperbook (Henze, Naceur, Nejdl, & Wolpers, 1999) offers individualized 
navigation support to the project resources based on the learners’ knowledge 
level and/or learning goals. Several systems focus on the collaborative perspec- 
tive of PBL and provide tools, such as NetPBL (Lee & Tsai, 2004), HyperNews 
(Hakkinen, 2002), iExpeditions (Wang, 1999), and Connection Log (Belland, 
2010), that support communication and interaction. In particular, NetPBL 
supports PBL in a blended learning context. It supports students’ interaction 
and idea/ artefact sharing. Students are allowed to (a) carry on synchronous 
and asynchronous discussions for the subquestions arising from the “driv- 
ing question” of the project or for several issues under investigation, and (b) 
upload and share their artefacts. HyperNews is a collaborative discussion 
environment enabling communication among students, teachers, and experts 
at the planning and evaluation phases of the project work. Connection Log 
is a Web-based system designed to scaffold middle school students’ creation 
of evidence -based arguments during problem-based units, allowing them 
to work individually and in collaboration. The above environments provide 
communication tools that support interaction among learners or between 
learners and tutors, as well as scaffolds that cultivate specific skills, at specific 
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phases of a project in a blended learning context that combines face-to-face 
learning with e-learning. Designing computer-based scaffolds that may 
provide support at the individual or group level needs to be reconsidered in 
a project-based e-learning context, raising questions about student needs, 
necessary metacognitive knowledge and skills, the way collaborative work 
might be orchestrated, and the design variables that should be considered. 

Following this line of research, we have developed MyProject (Papaniko- 
laou & Grigoriadou, 2009) as a Web-based adaptive learning environment 
that aims to support learners in progressively understanding the implicit 
issues of a project and accordingly organizing their work. To this end, My- 
Project gives learners a set of learning activities organised as a learning cycle 
and assists them in following their own paths through the cycle by offering 
adaptive guidance. The environment also provides hypermedia educational 
content composed of authentic cases as well as data about learners’ progress 
and interaction. At the group level, learners publish their own ideas and 
solutions when working on activities or studying the content. 

MyProject has been recently extended with a peer assessment functional- 
ity as an alternative authentic evaluation approach appropriate for PBL envi- 
ronments (Boubouka, Papanikolaou, & Grigoriadou, 2008). This functional- 
ity enables students at specific time intervals to submit their initial products, 
receive peer products for review, read their peers’ comments, make revi- 
sions, and resubmit their final products. 

As students’ metacognitive and reflective skills need to be strengthened 
in PBL environments, and metacognitive knowledge, such as self and task 
knowledge, strategic knowledge, and knowledge of plans and goals, is best 
supported in social settings for learning (McLoughlin & Hollingworth, 
2001), we further investigate in this paper the orchestration of collaboration 
in order to enhance metacognitive knowledge in a PBL context.To this end, 
we conducted an empirical study using a collaborative learning script com- 
bining individual and collaborative activities at specific phases of a project 
as an additional scaffold. We used MyProject in an e-learning context where 
all the interactions take place online and the life cycle of a project is inher- 
ent in the environment. This work combines research from the areas of PBL, 
metacognition, and computer- supported collaborative learning. 

In sum, the paper focuses on the following questions: 

• What are the design variables for a collaborative script intended to 
promote metacognitive knowledge? 

• What types of metacognitive knowledge should be better supported at 
specific phases of a project? 

• Which phases of a project should be better supported by peer learning? 

Research Context and Aims 

Critical issues in the design of a PBL environment are project selection, 
matching student requirements with task requirements, and the integration 
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of support and supplementary material with the project (Helle, Tynjala, & 
Olkinuora, 2006). In the process of designing support for students in a PBL 
environment, a valuable resource is research findings about metacogni- 
tion. Metacognition is used to describe peoples knowledge and regulation 
of human cognition (Flavell, 1979). Knowledge of cognition refers to what 
we know about our cognition. In particular, metacognitive research em- 
phasizes the need to balance cognitive and social competence as well as the 
need to create social, interactive, and reflective environments with a holistic 
approach to supporting metacognition (Lewis, 1998). Social interaction 
promotes the development of individuals’ cognitive structures as individuals 
reconcile differences between their own ideas and the ideas of others (see 
O’Donnell & King, 1999, for reviews of these theories), ask questions, and 
explain their reasoning for solutions (O’Donnell & King, 1999; Teasley, 1999; 
Teasley, 1997; Webb, 1992). O’Donnell and Kelly (1994) strongly suggest that 
work in groups provides opportunities for the development of members’ 
cognitive structures and cultivates positive attitudes toward the task and 
stronger task motivation compared to individual work. However, research 
on collaborative learning has demonstrated that its effectiveness depends on 
the richness and intensity of interactions among the group members (Dil- 
lenbourg, Baker, Blaye, & O’Malley, 1996). Two different classes of scaffolds 
have been distinguished to promote collaborative learning: those that pro- 
vide support on a conceptual level and those related to the interaction pro- 
cesses (Kollar, Fischer, & Hesse, 2006). The latter category usually structures 
the interaction process by shaping collaboration and assigning roles and 
specific activities to be carried out in a particular sequencing. Such scaffolds 
in the area of computer-supported collaborative learning (CSCL) are known 
as collaboration scripts (Dillenbourg & Hong, 2008; Kollar, Fischer, & Hesse, 
2006). Collaboration scripts can vary in the degree of freedom they attribute 
to learners to structure their collaboration. 

To this end, we investigate the introduction of collaborative tasks in spe- 
cific phases of a project in a way that promotes metacognitive knowledge. At 
specific stages of the project work, peer learning opportunities are combined 
with individual activities as an additional scaffold that aims to enhance 
sharing of individual reflective thinking with the group and consequently 
increase the quality of interaction. 

Currently, MyProject supports learners in working independently on a 
project and constructing their own knowledge. It promotes peer interaction 
by allowing learners to publish their ideas and solutions at specific phases 
of a project and to comment on them. MyProject provides scaffolds on 
the conceptual level as well as on the process level of completing a project. 
Because learners often feel lost in a project-based context, MyProject sup- 
ports learners in facing the challenges posed by organizing specific activities 
that gradually lead to the completion of the project in a “learning cycle” and 
provides adaptive guidance based on learners’ interaction behavior to assist 
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Table 1 : Learning Cycle Consisting of Four Stages with Available Scaffolds 


Stages 


Aims 


Introduction: 

An open-ended project is 
proposed. 


Enhance learners’ (a) knowledge of plans 
and goals as they try to understand, 
clarify, and define the problem(s) that the 
project poses, (b) metacognitive knowl- 
edge about the task demands and goals 


Generate Ideas: 

One or more “driving questions" 
are proposed in association with 
the expected learning outcomes. 


Cultivate metacognitive knowledge about 
the person (i.e., self-knowledge, strategic 
knowledge), as they elaborate on specific 
questions that stimulate conscious think- 
ing about the key points of the project 


Multiple Perspectives and 
Research: 

Educational content consisting 
of authentic cases is available 
in hypermedia form (Kolodner & 
Guzdial, 1999). 


Enhance learners’ metacognitive 
task knowledge and in particular that 
subcategory connected to the informa- 
tion available as students explore the 
hypermedia structure of the content and 
study and evaluate resources while trying 
to understand the type and quality of 
available information (Flavel, 1 9/9). 


Solution and Evaluation: Enhance learners’ metacognitive self, 

task, and strategic knowledge as they 

Peer assessment approach need t0 assess an( j rev j se their own and 

peers' work. 


Activities/Scaffolds 

Learners need to define their project or 
specific aspects that are intentionally open. 
Along with the open project, initial resources 
are also provided. 


Learners need to submit their perspectives 
(initial and final) to one or more driving 
questions. 

Links to corresponding educational material 
are provided next to each driving question. 
Access is allowed to (a) learners’ initial and 
final answers, (b) peers' contributions (after 
the initial submission), along with comments 
coming from peers. 

Opportunities for self-assessment are 
embedded in the content. 

Peers' contributions to self-assessment 
tasks as well as the teacher’s proposal are 
available. 

Graphical annotation augments the hyper- 
media structure of the content, reflecting the 
type of content and learners’ progress. 

A peer review activity encourages learners 
to submit a solution, then evaluate peers’ 
solutions, provide review comments, revise 
their work based on the reviews they receive, 
and make a final submission. 


them in moving through the cycle (Papanikolaou & Grigoriadou, 2009). The 
design of the learning cycle is inspired by the work of Schwartz, Lin, Brophy, 
and Bransford (1999) on STAR.LEGACY, which provides a visual represen- 
tation of a generic process that reflects the circular nature of understanding 
and the interaction of action, reflection, and revision within the learning 
process. 

In MyProject, the learning cycle consists of the stages presented in Table 
1, with each stage involving learning activities that cultivate different types 
of metacognitive knowledge. Various scaffolds are provided at each stage to 
help students progressively develop knowledge and skills and become able to 
complete the project. 

Although the learning cycle proposes a specific sequencing of learning 
activities, this sequence is indicatory because learners are allowed to freely 
navigate through the stages of the cycle. Figure 1 illustrates two screenshots 
of MyProject. On the top is a screenshot from the Generate Ideas stage, in 
which learners are encouraged to submit, in the corresponding text boxes, 
their initial and final responses to the driving questions. On the bottom is 
a screenshot from the Multiple Perspectives and Research stage, in which 
learners are encouraged to answer assessment questions included in the 
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Figure 1. Screenshots of MyProject at the Generate Ideas stage (top) and at the Multiple Perspectives and Research stage 
(bottom). 


content and submit their answers in the corresponding text box. In both cases, 
after students submit their initial proposals, they are allowed to access their 
peers’ contributions. The graphic representation of the learning cycle appears 
on the left of both screenshots as a spiral whose parts represent the various 
stages, with a tiny man accompanying the current stage. Learners may visit a 
particular stage by clicking on the corresponding part of the spiral. 

Method 

This empirical study adopted a collaboration script for the planning and 
evaluation phases, as these are the most critical for the project work. In 
the particular script, individual activities precede the collaborative ones to 
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promote sharing of individual reflective thinking during the collaborative 
activities. Design variables that we considered in this study are the roles that 
students undertake, the type of activities they perform, the type of interaction 
promoted at particular stages of the learning cycle, and activity sequencing. 

The study also explores how the following types of metacognitive knowledge, 
which are considered necessary for accomplishing a project, develop through 
the planning and evaluation phases (White, 1999): (a) self-knowledge, which 
entails individuals’ capacity to recognise their strengths/ weaknesses and evalu- 
ate themselves; (b) knowledge of plans and goals, referring to learners capacity 
to set and maintain goals and to record what they intend to do through their 
learning; (c) task-knowledge, which involves understanding the demands of 
tasks and what they require; and (d) strategic knowledge, which refers to the 
knowledge of usefulness of strategies available for achieving learning goals. 

Eighty-two (82) students attending a course titled Computer Science 
Education during the spring semester of the academic year 2007-2008 in the 
Computer Science and Telecommunications Department at the University of 
Athens participated in this study. The authors worked on the design, imple- 
mentation, and evaluation of the study, and a third researcher was involved 
at the implementation phase supervising the forums as well as at the data 
collection and analysis phases. The whole study lasted about one month. 
Initially students were randomly assigned to groups, but many of them 
opposed this approach. Most of these students doubt the effectiveness of a 
randomly formulated group and insist on working with fellow students they 
had worked with in the past. So we decided to allow these students to choose 
their group. Thus, most students were organised in groups randomly (42 
students; 51%), while 24 students (29%) chose their group, and 16 students 
(20%) experienced both situations due to several dropouts during the study. 

Students worked on a project about computer programming using the 
learning environment of MyProject, and group discussions took place in the 
forum of the courses e-class environment. The particular project was part of 
the course, and students’ performance on the project counted as 20% of their 
final course grades. We informed students about the collaboration script 
they had to follow through text-based descriptions and instructions. The 
time schedule was quite strict in order to make the work focus on particular 
topics at specific time intervals and promote participation in the flexible 
environment of an asynchronous forum. 

Students worked individually and in groups, undertaking multiple roles 
with increasing complexity through the various stages of the learning cycle. 
Table 2 (p. 142) presents the collaboration script, which is structured in two 
parts that correspond to the planning and evaluation phases of a project. 
Each part presents the activities that students performed along with the 
corresponding stage of the learning cycle, the level of the activity (indi- 
vidual, group, or class), and the duration. Following the particular script, 
students initially work as learners undertaking a project about computer 
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Table 2: Collaboration Script for Project-Based Learning 


Stage 

Level 

Activity 

Duration 

First Part: Planning Phase 

Introduction 

Individual 

Each student is informed about the project and explores initial resources. 


Generate Ideas 

Individual 

Students answer the main driving questions of the project that represent the 
expected learning outcomes, study corresponding educational content (if 
necessary), and read their peers’ contributions. 

5 days 

Multiple Perspectives 
& Research 

Individual 

Students study the hypermedia content provided to deal with the different 
issues/concepts involved in the project 



Group 

Students work in groups to discuss at the forum of the e-class the open 
issues of the project and agree on specific goals to attain (project definition) 

2 days 

Second Part: Evaluation Phase 




Individual 

Students submit their own solutions along with the definition of the project 
determined by the group 

3 days 

Solution & Evaluation 

Group 

Students work in groups to establish assessment criteria for the final project 
product 

3 days 


Individual 

Students review the solutions proposed by the rest of the members of their 
group based on the commonly agreed-upon assessment criteria 



Group 

Students work in groups to develop a common solution to the project 

6 days 


Individual 

Students revise their own solutions to the project based on the review com- 
ments they received from peers 



programming, and then as exp erts/revi ewers proposing assessment criteria 
and reviewing peer products. The script allows several degrees of interac- 
tion to encourage peer learning, including collaborative activities, sharing, 
and commenting on peers’ proposals submitted in a shared database at the 
Generate Ideas and Multiple Perspectives and Research stages. 

The script design takes into account the results of a pilot study of a forma- 
tive evaluation of MyProject during the spring semester 2006-2007 (Boubou- 
ka, Papanikolaou, & Grigoriadou, 2008). In this pilot study, 40 subjects worked 
individually on a project using a discussion forum as a helpdesk at the class 
level. After analysing the messages posted to this forum, we found that partici- 
pation was very limited. Out of 59 total posts, 29 (about 50%) referred to the 
project work and the remaining 30 referred to technical issues about MyPro- 
ject. These findings provided evidence about the need to orchestrate collabora- 
tion to increase interaction. 

In this particular script, individual activities precede collaborative activi- 
ties in the planning phase of the project to allow students to become in- 
formed about the open issues of the project and study so that they could ac- 
quire a common background before going to a collaborative session, where 
they should decide on the main goals of their project. These activities aim 
to promote students’ (a) self-knowledge, as they need to assess what they 
already know and what aspects they need to learn more about to deal with 
the project, and (b) knowledge of plans and goals as well as task knowledge, 
as the driving questions inform them about the learning goals of the project 
and they start thinking and discussing how they would face them. 
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At the evaluation phase of the project, students perform a peer assess- 
ment activity structured in several substages, in which a combination of 
individual and collaborative activities is proposed. Students submit their 
own solutions to the project as individual products and then act as review- 
ers/experts, discussing assessment criteria (collaborative activity) and 
reviewing peer products (individual activity). Students then participate in a 
collaborative activity in which they reflect on the different approaches they 
observed and clarify advantages/disadvantages by discussing with those who 
propose the alternative approaches. During this process, students reconsider 
their own solutions and collaboratively develop the “best” common solution. 
These activities aim to promote students’(a) self-knowledge, as they need to 
explain and argue for the strategies they use and comment on their peers’ 
ideas, arguments, and strategies; and (b) strategic knowledge, as they define 
and use specific criteria for reviewing project products, reflect on different 
approaches, and evaluate alternative strategies to build a common project 
product. 

At the end of the script, students reflected on their learning experiences and 
individually completed an evaluation questionnaire. The study finished with a 
final meeting, in which students and researchers discussed the learning expe- 
rience at the class level, focusing on advantages and possible improvements. 

Data Collection and Analysis 

The data we collected from the study and analysed included group discus- 
sions consisting of the messages that students posted at the e-class forum 
through the collaborative tasks, individual and group products/solutions, 
and the learners’ evaluation questionnaires. This paper focuses on the 
analysis of group discussions and the evaluation questionnaires that reflect 
students’ views. 

In particular, we performed quantitative and qualitative analyses of the 
messages posted at the asynchronous forum to explore the type of group dis- 
cussions students performed and the metacognitive knowledge they devel- 
oped through the discussions. Analysis of the group discussions was based 
on the interaction analysis model of Gunawardena, Lowe, and Anderson 
(1997), which examines the social construction of knowledge in computer 
conferencing. In particular, it is based on the grounded theory and uses the 
phases of a discussion to determine the amount of knowledge constructed: 

• Phase I: Sharing and comparison of information including (a) state- 
ments of observation or opinion, and (b) statements of agreement, 
examples, clarifications, and identifications of problems (C 1 category) 

• Phase II: Discovery and exploration of dissonance or inconsistency 
among ideas, concepts, or statements, including identification of ar- 
eas of disagreement and questions/answers to clarify the source and 
extent of disagreement (C2 category) 
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Figure 2. Types of messages exchanged between the members of seven groups for the definition of the project. 


• Phase III: Negotiation of meaning/co-construction of knowledge, 
including negotiation or clarification of the meaning of terms, 
identifications of areas of agreement, and negotiation of the 
relative weight to be assigned to types of argument (C3 category) 

• Phase IV: Testing and modification of proposed synthesis or co- 
construction, including testing the proposed synthesis against 
“received fact” and testing against existing cognitive schema, 
experiences, and literature (C4 category) 

• Phase V: Statements of agreement/application of newly constructed 
meaning, including summarization of agreement and applications 
of new knowledge and metacognitive statements revealing new 
knowledge construction (C5 category) 

We categorised each message posted at the forum according to the 
above scheme, which we extend with a C6 category to include messages 
about procedural issues concerning the tasks involved or requests about 
deadlines, technical issues, and social issues concerning interpersonal 
communication and organisation of group work. 
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Lastly, we analysed students’ answers to the evaluation questionnaire in 
order to explore students’ attitudes about (a) the type of tasks they per- 
formed through the various stages of the project (i.e., individual or col- 
laborative tasks) and (b) the usefulness of the peer learning opportunities 
provided. 

The questionnaire consisted of Likert scale questions (a sample of ques- 
tions appears in Appendix, pp. 154-155, where each row presents a question 
and each column shows the percentage of students who gave the specific 
answer). The questionnaire also included open-ended questions to encour- 
age students to argue for their selections. 

We based the questionnaire’s construction on the student interviews 
conducted the previous academic year for the pilot study. We used students’ 
comments about the way they worked through the various stages of the cycle 
to formulate the questions included in this questionnaire. 

Results 

Students collaborated in groups at specific stages of the learning cycle 
through an asynchronous discussion forum with 2,255 total messages. 

Below we evaluate the discussions the members of each group had about (a) 
the project definition, (b) assessment criteria for students’ projects, and (c) 
development of a group product. 

At the first part of the script (Planning phase), after students had worked 
individually to answer the initial driving questions of the Generation Ideas 
stage and had studied the content provided at the Multiple Perspectives and 
Research stage, they finally discuss in groups the open issues of the project 
in order to agree on specific goals and collaboratively define the project. 

The messages posted at the asynchronous forum provide evidence about 
the value of collaboration at this stage in promoting knowledge of plans 
and goals. 

In the discussion about the project definition, 29 groups participated and 
867 messages were posted. Although the type and amount of communication 
differs from group to group, the discussion developed to a much larger extent 
than was initially expected. In particular, the 29 groups discussed issues con- 
cerning the definition of their project as well as an outline of their project and 
the goals they would try to attain, whereas 25 groups went beyond that and 
discussed about how to solve the problem(s) posed, proposing and discuss- 
ing alternative strategies and work plans and providing relevant resources to 
strengthen their arguments. The type of discussions they developed were in 
the Cl, C3, and C5 categories, providing evidence about students’ participa- 
tion and elaboration on the goals of their project as well as the evolution of 
the group work to a final subproduct, the project definition (see Figure 2 for a 
sample of groups that exchanged a total of 262 messages; see Figures 2-4 for 
an illustration of the data from these groups). We also observed C2 discus- 
sions reflecting disagreements in several groups. Disagreements initiated a 
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Figure 3. Types of messages exchanged between the members of seven groups for the definition of assessment criteria. 


new round of negotiation, usually resulting in group agreement. Discussions 
in the C4 category were also slightly developed. Testing was likely not a target 
of the discussion due to the nature of the particular activity. In sum, most of 
the discussions were on negotiations that included thinking aloud about alter- 
native solutions and clarification of issues arising in the interest of reaching a 
consensus. This elaboration was obviously enhanced by their previous experi- 
ence working individually at the Generate Ideas and Multiple Perspectives and 
Research stages as well as by the knowledge they acquired, as students in many 
cases reflect on the driving questions to strengthen their arguments about spe- 
cific issues that need to be clarified in the project definition. Task knowledge 
also seemed to be promoted through this combination of individual and group 
activities, as in the process of defining the project, students decide on specific 
goals they would try to attain and they start thinking and discussing how this 
might happen. 

At the second part of the script and especially at the Solution and Evaluation 
stage, the messages posted provide evidence about the value of collaboration in 
promoting self and strategic knowledge. In particular, at this stage 26 of the 29 
groups participated, as members of the other three groups dropped out. 

Initially, the 26 groups worked to define assessment criteria for the final proj- 
ect product. Students proposed criteria for assessing the subject matter knowl- 
edge as well as programming and problem-solving skills. It was interesting how 
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the discussions evolved, due to subsequent contributions, from criteria focusing 
on specific programming structures to the quality of the algorithm, readability, 
and usability issues. They also discussed the assessment framework, including 
grading procedures and the type of assessment that should be adopted, as 
well as the value of quantitative versus qualitative assessment. Students seem 
to doubt the effectiveness of any grading procedure compared to qualitative 
assessment and feedback. The total number of messages posted was 616. 

In this activity, students developed Cl, C2, C3, and C5 types of discussion, 
providing evidence about their engagement in the evaluation process (see 
the sample of seven groups in Figure 3). Students also developed C4 discus- 
sions, as in many cases students suggest assessment criteria and test their 
validity by using them in hypothetical cases or in assessing their products. 
This seems to promote self-evaluation, as in many cases students apply the 
criteria they propose to their own solutions and share this information with 
the rest of the members of their group, which enhances their self-knowledge. 

Twenty-two of the 26 groups then held discussions to synthesize a com- 
mon solution to the project. At this substage, students discussed alternative 
solutions to develop a common product and at the same time revise the 
individual solutions they initially submitted. In this process, they discussed 
and compared alternative strategies to try to identify the “best” approach. 
This process was enhanced by the previous individual work, as the students, 
through the discussion, reflected on their own products and provided argu- 
ments based on their previous experience of reviewing peer solutions. The 
total number of messages exchanged decreased to 453; the participation of 
seven groups in this discussion is illustrated in Figure 4 (p. 148). 

The collaboration patterns adopted for this final submission are quite 
interesting: 22 of the 26 groups worked on the best solution among those 
proposed by the members of their groups and made refinements (to a 
smaller or larger extent) based on the peer reviews they had already devel- 
oped or received for their own solutions as well as the discussion among 
the group members to clarify the advantages and problems of alternative 
approaches. Only two groups did not manage to come to a consensus about 
the group product, and each member of those groups submitted a revised 
version of his/her initial submission. Two other groups decided to synthesize 
the best ideas of their members’ individual solutions and submit a com- 
mon final product. In many cases, students test their solutions or alternative 
ones provided by peers in authentic programming environments in order to 
synthesize the “best” solution out of the individual products, which led to 
C4 discussions. Students also try to revise their own solutions based on the 
feedback comments they received and ideas from their peers’ approaches. 
They usually discussed strategies and clarify several issues in order to use 
them in their own products or the common one. This process cultivated 
their strategic knowledge. Even in the cases in which students adopted 
one of the solutions, it is quite interesting that they also tried to integrate 
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Figure 4. Types of messages exchanged between the members of seven groups for synthesizing a common product. 


ideas from the rest of the individual products. In this process, they reached 
conflicts or knowledge deficiencies that they tried to clarify and overcome, 
leading to a small C2 discussion reflecting disagreements. 

In sum, students constructed the “best” solution in most cases gradually 
as they suggested and tested various approaches until submitting the final 
product. This socially shared reflective thinking about alternative solutions 
promotes self-knowledge as students try to understand, evaluate, and inte- 
grate them into their own products or a final group product. 

At the end of the script, learners completed an evaluation questionnaire 
assessing their learning experience through the various stages of the scenario 
(see Appendix, pp. 154-155). Eighty-three students completed the question- 
naire. Students’ answers and comments provide evidence about the value of 
the mode of work proposed (a combination of individual and collaborative 
tasks) to support them in completing the project. Specifically, most students 
agree (and strongly agree) that group collaboration through the forum was 
quite supportive for defining the project and planning their work (see Ql: 
27%+57%), whereas only 6% of the students seem to have faced difficulties 
in this group process (see Q2: 2%+4%). Regarding students’ responses to an 
open-ended question about their preferable way of working at the Introduc- 
tion stage, 70% prefer the group work, whereas 15.5% would prefer alternative 
ways of collaboration, such as face-to-face communication or chat, teacher 
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participation in the discussion, or collaboration among groups. It is remark- 
able that only two students (2%) were against collaboration, preferring the 
teacher to provide a clear project definition. 

Concerning students’ attitudes about the peer learning opportunities pro- 
vided at the Generate Ideas stage, most of them agree (and strongly agree) 
that the option of submitting their proposal to a database accessible by peers 
was very helpful (see Q3:52%+37%). In particular, students commented that 
publishing their proposals in a database stimulates them to clearly explain 
their ideas and promotes reflection and critical thinking. Moreover, students 
also believe that reading the ideas of others as well as commenting on them 
stimulates consideration of how the ideas of others work and is quite helpful 
(Q4: 42%+43%). According to the students’ comments, 75% of them took 
advantage of the possibility of sharing their ideas with peers at the Gener- 
ate Ideas stage by looking at their peers’ answers, and 43% commented on 
them. As far as the mode of work at the Generate Ideas stage is concerned, 
students’ preferences are divided: 53% seem to prefer collaborative work for 
the final submission to the driving questions, whereas 47% prefer individual 
work at this stage (as they worked through the study) (see Q5). 

As far as the Multiple Perspectives and Research stage is concerned, most 
students seem to easily find their own way through the educational content. 
At this stage, 62% of the students prefer individual work when studying the 
content. However, most of them seem to appreciate the option of sharing 
their answers to assessment tasks (see G6, Q7), and 51% of the students 
suggest that group work would be helpful for completing the assessment 
tasks (see Q8: 22%+29%). A majority (67%) of students took advantage of 
the possibility of sharing their solutions to assessment tasks at the Multiple 
Perspectives and Research stage by looking at their peers’ answers, but only 
24% provided comments. 

Students’ attitudes about the mode of work adopted at the Solution and 
Evaluation stage (i.e., the particular combination of individual and collabor- 
ative tasks) are positive, although various other proposals were also submit- 
ted. Seventy-nine percent of the students found that working individually to 
construct an initial solution was quite supportive for the learning process, 
while 9% of the students disagree (see Q9). Students found collaboration for 
the definition of assessment criteria helpful for the assessment process and 
for improving their own solutions (see Q10: 33%+56%). Most students re- 
ported that their group easily managed to synthesize a common product (see 
Q1 1: 36%+48%). In this process, as illustrated in Table 3 (p. 150), they also 
appreciate as valuable and supportive the different options of peer support, 
including (a) group collaboration for defining assessment criteria (89%), (b) 
evaluating their peer’s solutions (88%), (c) receiving feedback comments on 
their work from peers (85%), and (d) group collaboration for the submission 
of a final group product (85%). Only 6% of the students seemed to find no 
value in these processes. 
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Table 3: Students’ Attitudes about the Effectiveness of Various Types of Peer Support 



Positive 

Neutral 

Negative 

Group work for assessment criteria 

73 (89%) 

5 (6%) 

4 (5%) 

Evaluating peers’ solutions 

72 (88%) 

6 (7%) 

5 (6%) 

Receiving/reviewing comments from peers 

70 (85%) 

6 (7%) 

5 (6%) 

Group work for final group product 

70 (85%) 

6 (7%) 

5 (6%) 


As far as the mode of work at this stage is concerned, students suggested 
alternative approaches. Forty-nine students (60%) prefer working the way 
they were asked to (i.e., individually at first and then in groups for the 
construction of a final group product), whereas only two students (2.5%) 
prefer to work in groups at first and then individually. Ten (12%) students 
would prefer to work in groups throughout the Solution and Evaluation 
stage instead of the final part of the stage when submitting a group product, 
whereas ten students ( 12%) argue that the final product should be individu- 
ally constructed based on the revised version of their initial product. Fur- 
thermore, four students (5%) would prefer teacher guidance instead of peer 
interaction. It is remarkable that most of the students prefer to collaborate at 
this stage, and none of them prefers to work individually. 

Generally, students seemed to appreciate the value of interaction and col- 
laboration opportunities at various stages of the MyProject learning cycle to 
complete the project. Among the benefits of collaboration, students ac- 
knowledge that collaboration (a) promoted the exchange of many different 
ideas and opinions (35 students, 43%), (b) supported the establishment of 
common goals (15 students, 18%), and (c) enhanced peer learning as they 
helped each other to overcome difficulties (11 students, 13%). The students 
attitudes about the importance of the goals they attained through the project 
include appreciation of (a) elaboration on the subject matter (58%), (b) 
collaboration (79%), (c) ICT use (59%), and (d) engagement in the project 
process (80%). 

Other benefits that students appreciated highly were the sense of commu- 
nity that the collaborative tasks promoted and the immediate feedback they 
usually received from peers. Students also liked the combination of individual 
and collaborative tasks and the multiple roles they undertook. One student 
commented, “In a few days you pass from the role of a student to the role of a 
teacher.” Members of groups that did not manage to productively collaborate 
also outlined negative attributes of group work, such as lack of communication 
or irresponsibility of particular members of the group. 

Discussion and Future Plans 

This empirical study investigated the value of collaboration in promoting 
the sharing of individual reflective thinking in group work and enhancing 
metacognitive knowledge in a project-based e-learning context. Group discus- 
sions provide evidence about the different types of metacognitive knowledge 
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that different stages of the project work cultivated, as well as the influence of 
individual work on group reflective thinking and of group work on individual 
development. Moreover, students who participated in the study seem to value 
the opportunities for collaboration and peer interaction it offered, although 
the individual tasks were also considered necessary at specific stages of the 
project. They value them for promoting task motivation as well as facilitating 
inquiry, understanding, and application of domain concepts involved in the 
particular project. Below we discuss the main questions under investigation. 

What Are the Design Variables for a Collaborative Script Intended 
to Promote Metacognitive Knowledge? 

The main design variables of the script are: project phase, roles undertaken 
by students, degree and type of interaction, type of activities and products, and 
activity sequencing. In this script, students undertake multiple roles with a 
gradual increase of cognitive load, and the last part of the script uses a mecha- 
nism for role turntaking as they become authors, then reviewers, of peer prod- 
ucts and finally authors of a group product. In this particular script, students 
were involved at the planning and evaluation phases of the project, initially 
in individual activities and then in collaborative ones. The degree and type 
of interaction among students differ at the various stages of the project (i.e., 
degree of interaction is considered low when students are allowed to see and 
comment on their peers’ submissions but not to directly interact with them). 
Types of interaction that this script used are (a) sharing through a database 
accessible by peers, (b) reviewing peer products, (c) collaborating to attain a 
common goal, and (d) collaborating to develop a common product. The evalu- 
ation of the types of discussion performed reveal that the type of activities 
(individual vs. collaborative activities) that students undertake, the products 
that they need to construct, and the activity sequence all influenced the type of 
discussions they developed. For example, in the case of groups collaborating 
on the development of a group product, students spent considerable effort on 
discussing, testing, and trying out alternative approaches in authentic pro- 
gramming environments with the goal of synthesising a common solution. 
Moreover, the students’ previous individual work as authors and reviewers of 
peer products influenced the discussions greatly. In the case of groups working 
to find a definition for their project, the previous individual tasks of answering 
the driving questions at the Generate Ideas and studying the content seem to 
have influenced students’ proposals and contributions to the discussion. We 
also observed that the groups with a high communication rate (i.e., posted 
many messages) gradually developed discussions in the categories C2, C3, 

C4, and C5. However, the discussions of those groups that did not manage to 
collaborate was restricted to opinion sharing and practical issues. Especially 
at the Solution and Evaluation stage, we expected the discussions to develop 
much further, as students had to negotiate the establishment of quality criteria 
and construct a common solution. Based on the collaboration patterns that 
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the groups adopted, only 2 of the 26 groups collaborated at the Solution and 
Evaluation stage in order to integrate the individual solutions to one final ver- 
sion. The cognitive load of this process is much heavier compared to the ap- 
proach adopted by the rest of the 24 groups that decided to submit the “best” 
of the individual solutions with minor or major revisions. 

An interesting issue to further investigate is the synthesis of the groups in 
terms of individual members’ characteristics, such as prior knowledge, style, 
preferences, and how group synthesis relates to the effectiveness of collabo- 
ration or their preference for different modes of collaboration. Moreover, 
interaction among different groups would be another valuable parameter to 
further investigate as an alternative intervention for increasing the quality of 
interaction within a group. 

What Types of Metacognitive Knowledge Should Be Better Supported at 
Specific Phases of a Project? 

This script promotes knowledge of plans and goals as well as task knowledge at 
the planning phase. In this phase, after students had been informed about the 
underlying learning outcomes and had submitted their initial ideas on how to 
face the critical issues posed, the need to decide on the project’s specific pa- 
rameters stimulated discussion about which goals they would try to attain and 
how. Although students had access to educational content, their self-knowl- 
edge did not appear to increase, perhaps due to the fact that they were quite 
experienced programmers so they didn’t “need” to study in order to define 
the project. Self as well as strategic knowledge increased when students had to 
to develop and submit their own solutions at the evaluation phase, especially 
when they collaborated to develop a common product and compared their 
own work with alternative approaches adopted by peers. Although students 
had individually developed solutions and reviewed peer solutions with alter- 
native approaches, they seemed to realize the value of the strategies their peers 
used when they need to select the optimum solution for the final common 
product, and in this process they discussed the value of using theoretical evi- 
dence or providing testing results from authentic programming environments. 

Which Phases of a Project Should Be Better Supported by Peer Learning? 

Based on group discussions and on students’ attitudes about the peer learn- 
ing opportunities offered, most seem to appreciate the value of interaction and 
collaboration opportunities at various stages of the MyProject learning cycle for 
accomplishing the project. In the planning phase, including the first three stages 
of the learning cycle, collaboration was quite helpful for clarifying issues, defin- 
ing the project, and planning their work. Alternative ways of collaboration were 
also proposed that involved the teacher or other groups. Most students preferred 
to collaboratively elaborate on the driving questions instead of simply being 
informed about peer ideas. They seemed to prefer studying the content on their 
own, although they found it helpful to share their solutions to self-assessment 
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tasks with peers. In the evaluation phase, students found all the types of interac- 
tion involved quite helpful. Alternative modes of collaboration involving mainly 
collaboration/interaction among groups were also proposed. 

Building on the results of this study, we intend to further investigate how 
alternative collaboration scripts promote the development of individuals’ 
cognitive structures and particular types of metacognitive knowledge at each 
phase of the project work. 
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Appendix 


A sample of questions of the evaluation questionnaire provided to students. 

A Likert scale is used in which 2 = strongly agree, 1 = agree, 0 = undecided, -1 = disagree, -2 

= strongly disagree. 


Introduction Stage 

-2 

-1 

0 

1 

2 

Q1 : Do you believe that collaborating with peers for defining the project was supportive? 

1% 

1% 

14% 

27% 

57% 

Q2: Do you believe it was easy for your group to reach a common project definition? 

Generate Ideas Stage 

2% 

4% 

10% 

37% 

47% 

Q3: Do you believe that submitting your proposals to a database accessible by peers 
supports learning? 

0% 

2% 

8% 

52% 

37% 

Q4: Do you believe that the option of accessing and commenting on peers’ proposals 
supports learning? 

1% 

1% 

12% 

42% 

43% 

Q5: Do you believe that submitting your answers to the driving questions at the Generate 
Ideas stage, initially individually and finally in collaboration with your group, would be helpful? 

Multiple Perspectives & Research Stage 

8% 

17% 

22% 

16% 

37% 

Q6: Do you believe that submitting your solutions to assessment tasks to a database 
accessible by peers supports learning? 

6% 

4% 

8% 

45% 

37% 

Q7: Do you believe that the option of accessing and commenting on peers' solutions 
supports learning? 

2% 

5% 

10% 

45% 

38% 

Q8: Do you believe that group work through a forum would be helpful for completing 
the assessment tasks? 

Solution & Evaluation Stage 

18% 

8% 

23% 

22% 

29% 

Q9: Do you believe that submitting individually the initial solution to the project was 
supportive for learning? 

4% 

5% 

12% 

13% 

66% 

Q1 0: Do you believe that collaborating with peers for defining assessment criteria 
was supportive for improving your solution? 

2% 

2% 

6% 

33% 

56% 

Q1 1 : Did you manage to synthesize a group product easily? 

7% 

1% 

7% 

36% 

48% 
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